Multidimensional signal detection modeling reveals Gestalt-like perceptual integration of face emotion and identity.

Numerous studies have tested the hypothesis that facial identity and emotional expression are independently processed, but a solid conclusion has been difficult to reach, with the literature showing contradictory results. We argue that this is partly due to different researchers using different definitions of perceptual integration and independence, usually vague and/or simply operational, and also due to lack of proper stimulus control. Here, we performed a study using three-dimensional realistic computer-generated faces for which the discriminability of identities and expressions, the intensity of the expressions, and low-level features of the faces were controlled. A large number of participants, distributed across twelve experimental groups, performed identification tasks for the six basic emotional expressions and the neutral expression (between 2018 and 2019). A multidimensional signal detection model was utilized to analyze the data, which allowed us to dissociate between multiple formally defined notions of independence and holism. Results showed strong and robust violations of perceptual independence that were consistent across all experiments and suggest Gestalt-like perceptual integration of face identity and expression. To date, our results provide the strongest evidence for holistic/Gestalt processing found among face perception studies that have used formal definitions of independence and holism. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Perceptual Observer Modeling Reveals Likely Mechanisms of Face Expression Recognition Deficits in Depression.

BACKGROUND: Deficits in face emotion recognition are well documented in depression, but the underlying mechanisms are poorly understood. Psychophysical observer models provide a way to precisely characterize such mechanisms. Using model-based analyses, we tested 2 hypotheses about how depression may reduce sensitivity to detect face emotion: 1) via a change in selectivity for visual information diagnostic of emotion or 2) via a change in signal-to-noise ratio in the system performing emotion detection. METHODS: Sixty adults, one half meeting criteria for major depressive disorder and the other half healthy control participants, identified sadness and happiness in noisy face stimuli, and their responses were used to estimate templates encoding the visual information used for emotion identification. We analyzed these templates using traditional and model-based analyses; in the latter, the match between templates and stimuli, representing sensory evidence for the information encoded in the template, was compared against behavioral data. RESULTS: Estimated happiness templates produced sensory evidence that was less strongly correlated with response times in participants with depression than in control participants, suggesting that depression was associated with a reduced signal-to-noise ratio in the detection of happiness. The opposite results were found for the detection of sadness. We found little evidence that depression was accompanied by changes in selectivity (i.e., information used to detect emotion), but depression was associated with a stronger influence of face identity on selectivity. CONCLUSIONS: Depression is more strongly associated with changes in signal-to-noise ratio during emotion recognition, suggesting that deficits in emotion detection are driven primarily by deprecated signal quality rather than suboptimal sampling of information used to detect emotion.

Face shape and motion are perceptually separable: Support for a revised model of face processing.

A recent model of face processing proposes that face shape and motion are processed in parallel brain pathways. Although tested in neuroimaging, the assumptions of this theory remain relatively untested through controlled psychophysical studies until now. Recruiting undergraduate students over the age of 18, we test this hypothesis using a tight control of stimulus factors, through computerized three-dimensional face models and calibration of dimensional discriminability, and of decisional factors, through a model-based analysis using general recognition theory (GRT). Theoretical links between neural and perceptual forms of independence within GRT allowed us to derive the a priori hypotheses that perceptual separability of shape and motion should hold, while other forms of independence defined within GRT might fail. We found evidence to support both of those predictions.

Towards a mechanistic understanding of the role of error monitoring and memory in social anxiety.

Cognitive models state social anxiety (SA) involves biased cognitive processing that impacts what is learned and remembered within social situations, leading to the maintenance of SA. Neuroscience work links SA to enhanced error monitoring, reflected in error-related neural responses arising from mediofrontal cortex (MFC). Yet, the role of error monitoring in SA remains unclear, as it is unknown whether error monitoring can drive changes in memory, biasing what is learned or remembered about social situations. Thus, we developed a novel paradigm to investigate the role of error-related MFC theta oscillations (associated with error monitoring) and memory biases in SA. EEG was collected while participants completed a novel Face-Flanker task, involving presentation of task-unrelated, trial-unique faces behind target/flanker arrows on each trial. A subsequent incidental memory assessment evaluated memory biases for error events. Severity of SA symptoms were associated with greater error-related theta synchrony over MFC, as well as between MFC and sensory cortex. SA was positively associated with memory biases for error events. Consistent with a mechanistic role in biased cognitive processing, greater error-related MFC-sensory theta synchrony during the Face-Flanker predicted subsequent memory biases for error events. Our findings suggest high SA individuals exhibit memory biases for error events, and that this behavioral phenomenon may be driven by error-related MFC-sensory theta synchrony associated with error monitoring. Moreover, results demonstrate the potential of a novel paradigm to elucidate mechanisms underlying relations between error monitoring and SA.

Why is the Rescorla-Wagner model so influential?

The influence of the Rescorla-Wagner model cannot be overestimated, despite that (1) the model does not differ much computationally from its predecessors and competitors, and (2) its shortcomings are well-known in the learning community. Here we discuss the reasons behind its widespread influence in the cognitive and neural sciences, and argue that it is the constant search for general-process theories by learning scholars which eventually produced a model whose application spans many different areas of research to this day. We focus on the theoretical and empirical background of the model, the theoretical connections that it has with later developments across Marr's levels of analysis, as well as the broad variety of research that it has guided and inspired.

Improving the validity of neuroimaging decoding tests of invariant and configural neural representation.

Many research questions in sensory neuroscience involve determining whether the neural representation of a stimulus property is invariant or specific to a particular stimulus context (e.g., Is object representation invariant to translation? Is the representation of a face feature specific to the context of other face features?). Between these two extremes, representations may also be context-tolerant or context-sensitive. Most neuroimaging studies have used operational tests in which a target property is inferred from a significant test against the null hypothesis of the opposite property. For example, the popular cross-classification test concludes that representations are invariant or tolerant when the null hypothesis of specificity is rejected. A recently developed neurocomputational theory suggests two insights regarding such tests. First, tests against the null of context-specificity, and for the alternative of context-invariance, are prone to false positives due to the way in which the underlying neural representations are transformed into indirect measurements in neuroimaging studies. Second, jointly performing tests against the nulls of invariance and specificity allows one to reach more precise and valid conclusions about the underlying representations, particularly when the null of invariance is tested using the fine-grained information from classifier decision variables rather than only accuracies (i.e., using the decoding separability test). Here, we provide empirical and computational evidence supporting both of these theoretical insights. In our empirical study, we use encoding of orientation and spatial position in primary visual cortex as a case study, as previous research has established that these properties are encoded in a context-sensitive way. Using fMRI decoding, we show that the cross-classification test produces false-positive conclusions of invariance, but that more valid conclusions can be reached by jointly performing tests against the null of invariance. The results of two simulations further support both of these conclusions. We conclude that more valid inferences about invariance or specificity of neural representations can be reached by jointly testing against both hypotheses, and using neurocomputational theory to guide the interpretation of results.

Sensitivity vs. awareness curve: A novel model-based analysis to uncover the processes underlying nonconscious perception.

In this study, we present a novel model-based analysis of the association between awareness and perceptual processing based on a multidimensional version of signal detection theory (general recognition theory, or GRT). The analysis fits a GRT model to behavioral data and uses the estimated model to construct a sensitivity versus awareness (SvA) curve, representing sensitivity in the discrimination task at each value of relative likelihood of awareness. This approach treats awareness as a continuum rather than a dichotomy, but also provides an objective benchmark for low likelihood of awareness. In two experiments, we assessed nonconscious facial expression recognition using SvA curves in a condition in which faces (fearful vs. neutral) were rendered invisible using continuous flash suppression (CFS) for 500 and 700 milliseconds. We predicted and found nonconscious processing of face emotion, in the form of higher than chance-level sensitivity in the area of low likelihood of awareness.

Subsampling of cues in associative learning.

Theories of learning distinguish between elemental and configural stimulus processing depending on whether stimuli are processed independently or as whole configurations. Evidence for elemental processing comes from findings of summation in animals where a compound of two dissimilar stimuli is deemed to be more predictive than each stimulus alone, whereas configural processing is supported by experiments using similar stimuli in which summation is not found. However, in humans the summation effect is robust and impervious to similarity manipulations. In three experiments in human predictive learning, we show that summation can be obliterated when partially reinforced cues are added to the summands in training and tests. This lack of summation only holds when the partially reinforced cues are similar to the reinforced cues (experiment 1) and seems to depend on participants sampling only the most salient cue in each trial (experiments 2a and 2b) in a sequential visual search process. Instead of attributing our and others' instances of lack of summation to the customary idea of configural processing, we offer a formal subsampling rule that might be applied to situations in which the stimuli are hard to parse from each other.

A computational account of the mechanisms underlying face perception biases in depression.

Here, we take a computational approach to understand the mechanisms underlying face perception biases in depression. Thirty participants diagnosed with major depressive disorder and 30 healthy control participants took part in three studies involving recognition of identity and emotion in faces. We used signal detection theory to determine whether any perceptual biases exist in depression aside from decisional biases. We found lower sensitivity to happiness in general, and lower sensitivity to both happiness and sadness with ambiguous stimuli. Our use of highly-controlled face stimuli ensures that such asymmetry is truly perceptual in nature, rather than the result of studying expressions with inherently different discriminability. We found no systematic effect of depression on the perceptual interactions between face expression and identity. We also found that decisional strategies used in our task were different for people with depression and controls, but in a way that was highly specific to the stimulus set presented. We show through simulation that the observed perceptual effects, as well as other biases found in the literature, can be explained by a computational model in which channels encoding positive expressions are selectively suppressed. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Adaptation aftereffects reveal how categorization training changes the encoding of face identity.

Previous research suggests that learning to categorize faces along a novel dimension changes the perceptual representation of such dimension, increasing its discriminability, its invariance, and the information used to identify faces varying along the dimension. A common interpretation of these results is that categorization training promotes the creation of novel dimensions, rather than simply the enhancement of already existing representations. Here, we trained a group of participants to categorize faces that varied along two morphing dimensions, one of them relevant to the categorization task and the other irrelevant to the task. An untrained group did not receive such categorization training. In three experiments, we used face adaptation aftereffects to explore how categorization training changes the encoding of face identities at the extremes of the category-relevant dimension and whether such training produces encoding of the category-relevant dimension as a preferred direction in face space. The pattern of results suggests that categorization training enhances the already existing norm-based coding of face identity, rather than creating novel category-relevant representations. We formalized this conclusion in a model that explains the most important results in our experiments and serves as a working hypothesis for future work in this area.

FaReT: A free and open-source toolkit of three-dimensional models and software to study face perception.

A problem in the study of face perception is that results can be confounded by poor stimulus control. Ideally, experiments should precisely manipulate facial features under study and tightly control irrelevant features. Software for 3D face modeling provides such control, but there is a lack of free and open source alternatives specifically created for face perception research. Here, we provide such tools by expanding the open-source software MakeHuman. We present a database of 27 identity models and six expression pose models (sadness, anger, happiness, disgust, fear, and surprise), together with software to manipulate the models in ways that are common in the face perception literature, allowing researchers to: (1) create a sequence of renders from interpolations between two or more 3D models (differing in identity, expression, and/or pose), resulting in a "morphing" sequence; (2) create renders by extrapolation in a direction of face space, obtaining 3D "anti-faces" and caricatures; (3) obtain videos of dynamic faces from rendered images; (4) obtain average face models; (5) standardize a set of models so that they differ only in selected facial shape features, and (6) communicate with experiment software (e.g., PsychoPy) to render faces dynamically online. These tools vastly improve both the speed at which face stimuli can be produced and the level of control that researchers have over face stimuli. We validate the face model database and software tools through a small study on human perceptual judgments of stimuli produced with the toolkit.

Evidence for a dissociation between causal beliefs and instrumental actions.

Human experiments have demonstrated that instrumental performance of an action and the causal beliefs of the effectiveness of an action in producing a reward are correlated and controlled by the probability of an action leading to a reward. The animal literature, however, shows that instrumental performance under free-operant training differs even when the reward probabilities are matched while subjects undergo training under ratio or interval schedules of reward. In two experiments, we investigated whether causal beliefs would correlate with instrumental performance under ratio and interval schedules for matched reward probabilities. In both experiments, we found that performance was higher under ratio than under interval training. However, causal beliefs were similar between these two conditions despite these differences in instrumental performance. When reward probabilities were increased by experimental manipulations in Experiment 2, the causal beliefs increased but performance decreased with respect to Experiment 1. This is evidence that instrumental performance and causal action-reward attribution may not follow from the same psychological process under free-operant training.

Beyond the "Conceptual Nervous System": Can computational cognitive neuroscience transform learning theory?

In the last century, learning theory has been dominated by an approach assuming that associations between hypothetical representational nodes can support the acquisition of knowledge about the environment. The similarities between this approach and connectionism did not go unnoticed to learning theorists, with many of them explicitly adopting a neural network approach in the modeling of learning phenomena. Skinner famously criticized such use of hypothetical neural structures for the explanation of behavior (the "Conceptual Nervous System"), and one aspect of his criticism has proven to be correct: theory underdetermination is a pervasive problem in cognitive modeling in general, and in associationist and connectionist models in particular. That is, models implementing two very different cognitive processes often make the exact same behavioral predictions, meaning that important theoretical questions posed by contrasting the two models remain unanswered. We show through several examples that theory underdetermination is common in the learning theory literature, affecting the solvability of some of the most important theoretical problems that have been posed in the last decades. Computational cognitive neuroscience (CCN) offers a solution to this problem, by including neurobiological constraints in computational models of behavior and cognition. Rather than simply being inspired by neural computation, CCN models are built to reflect as much as possible about the actual neural structures thought to underlie a particular behavior. They go beyond the "Conceptual Nervous System" and offer a true integration of behavioral and neural levels of analysis.

Categorization training changes the visual representation of face identity.

Previous research suggests that learning to categorize faces along a new dimension changes the perceptual representation of that dimension, but little is known about how the representation of specific face identities changes after such category learning. Here, we trained participants to categorize faces that varied along two morphing dimensions. One dimension was relevant to the categorization task and the other was irrelevant. We used reverse correlation to estimate the internal templates used to identify the two faces at the extremes of the relevant dimension, both before and after training, and at two different levels of the irrelevant dimension. Categorization training changed the internal templates used for face identification, even though identification and categorization tasks impose different demands on the observers. After categorization training, the internal templates became more invariant across changes in the irrelevant dimension. These results suggest that the representation of face identity can be modified by categorization experience.

Novel representations that support rule-based categorization are acquired on-the-fly during category learning.

Humans learn categorization rules that are aligned with separable dimensions through a rule-based learning system, which makes learning faster and easier to generalize than categorization rules that require integration of information from different dimensions. Recent research suggests that learning to categorize objects along a completely novel dimension changes its perceptual representation, making it more separable and discriminable. Here, we asked whether such newly learned dimensions could support rule-based category learning. One group received extensive categorization training and a second group did not receive such training. Later, both groups were trained in a task that made use of the category-relevant dimension, and then tested in an analogical transfer task (Experiment 1) and a button-switch interference task (Experiment 2). We expected that only the group with extensive pre-training (with well-learned dimensional representations) would show evidence of rule-based behavior in these tasks. Surprisingly, both groups performed as expected from rule-based learning. A third experiment tested whether a single session (less than 1 h) of training in a categorization task would facilitate learning in a task requiring executive function. There was a substantial learning advantage for a group with brief pre-training with the relevant dimension. We hypothesize that extensive experience with separable dimensions is not required for rule-based category learning; rather, the rule-based system may learn representations "on the fly" that allow rule application. We discuss what kind of neurocomputational model might explain these data best.

Human instrumental performance in ratio and interval contingencies: A challenge for associative theory.

Associative learning theories regard the probability of reinforcement as the critical factor determining responding. However, the role of this factor in instrumental conditioning is not completely clear. In fact, free-operant experiments show that participants respond at a higher rate on variable ratio than on variable interval schedules even though the reinforcement probability is matched between the schedules. This difference has been attributed to the differential reinforcement of long inter-response times (IRTs) by interval schedules, which acts to slow responding. In the present study, we used a novel experimental design to investigate human responding under random ratio (RR) and regulated probability interval (RPI) schedules, a type of interval schedule that sets a reinforcement probability independently of the IRT duration. Participants responded on each type of schedule before a final choice test in which they distributed responding between two schedules similar to those experienced during training. Although response rates did not differ during training, the participants responded at a lower rate on the RPI schedule than on the matched RR schedule during the choice test. This preference cannot be attributed to a higher probability of reinforcement for long IRTs and questions the idea that similar associative processes underlie classical and instrumental conditioning.

Linking signal detection theory and encoding models to reveal independent neural representations from neuroimaging data.

Many research questions in visual perception involve determining whether stimulus properties are represented and processed independently. In visual neuroscience, there is great interest in determining whether important object dimensions are represented independently in the brain. For example, theories of face recognition have proposed either completely or partially independent processing of identity and emotional expression. Unfortunately, most previous research has only vaguely defined what is meant by "independence," which hinders its precise quantification and testing. This article develops a new quantitative framework that links signal detection theory from psychophysics and encoding models from computational neuroscience, focusing on a special form of independence defined in the psychophysics literature: perceptual separability. The new theory allowed us, for the first time, to precisely define separability of neural representations and to theoretically link behavioral and brain measures of separability. The framework formally specifies the relation between these different levels of perceptual and brain representation, providing the tools for a truly integrative research approach. In particular, the theory identifies exactly what valid inferences can be made about independent encoding of stimulus dimensions from the results of multivariate analyses of neuroimaging data and psychophysical studies. In addition, commonly used operational tests of independence are re-interpreted within this new theoretical framework, providing insights on their correct use and interpretation. Finally, we apply this new framework to the study of separability of brain representations of face identity and emotional expression (neutral/sad) in a human fMRI study with male and female participants.

Exploring the Effect of Stimulus Similarity on the Summation Effect in Causal Learning.

Several contemporary models anticipate that the summation effect is modulated by the similarity between the cues forming a compound. Here, we explore this hypothesis in a series of causal learning experiments. Participants were presented with two visual cues that separately predicted a common outcome and later asked for the outcome predicted by the compound of the two cues. Similarity was varied between groups through changes in shape, spatial position, color, configuration, and rotation. In variance with the predictions of these models, we observed similar and strong levels of summation in both groups across all manipulations of similarity. The effect, however, was significantly reduced by manipulations intended to impact assumptions about the causal independence of the cues forming the compound, but this reduction was independent of stimulus similarity. These results are problematic for similarity-based models and can be more readily explained by rational approaches to causal learning.

Contemporary associative learning theory predicts failures to obtain blocking: Comment on Maes et al. (2016).

In a recent article, Maes et al. (2016) report the results from 15 experiments in the blocking effect, all of which failed to replicate the basic phenomenon. Whereas Maes et al. did not dispute the reality of the blocking effect, they concluded that the effect is more difficult to obtain than what could be assumed from the literature and that we lack insight into its boundary conditions. This conclusion is incorrect, because contemporary associative learning theory both agrees with the authors' conclusion that blocking is parameter-dependent and it makes specific predictions about the experimental parameters likely to produce a small or no blocking effect. Ten of the 15 experiments presented by Maes et al. use exactly those parameters (same-modality stimuli for the compound AX), making their results completely unsurprising in the light of contemporary associative learning theory. The results from 3 other experiments are difficult to interpret because of a floor effect. A failure to replicate blocking in only 2 experiments is unsurprising and can be explained as the result of statistical variability or changes in experimental procedure. (PsycINFO Database Record

Testing Separability and Independence of Perceptual Dimensions with General Recognition Theory: A Tutorial and New R Package (grtools).

Determining whether perceptual properties are processed independently is an important goal in perceptual science, and tools to test independence should be widely available to experimental researchers. The best analytical tools to test for perceptual independence are provided by General Recognition Theory (GRT), a multidimensional extension of signal detection theory. Unfortunately, there is currently a lack of software implementing GRT analyses that is ready-to-use by experimental psychologists and neuroscientists with little training in computational modeling. This paper presents grtools, an R package developed with the explicit aim of providing experimentalists with the ability to perform full GRT analyses using only a couple of command lines. We describe the software and provide a practical tutorial on how to perform each of the analyses available in grtools. We also provide advice to researchers on best practices for experimental design and interpretation of results when applying GRT and grtools.